J. Scott Armstrong

J. Scott Armstrong is a Wharton professor whose research on forecasting and expertise supports a central argument of The Wisdom of Crowds: expertise beyond a minimal level often adds little to predictions of change, and decision-makers should pool expert judgment rather than chase an individual seer. Armstrong’s findings give the book’s case against expert authority an empirical basis, while preserving a role for informed specialists within a broader group.

Research on Forecasting

Armstrong surveyed expert forecasts and analyses across a wide range of fields and reported that he could find no studies showing an important advantage for expertise. Some experts forecast slightly better than laypeople, but the advantage did not grow reliably with expertise: Armstrong concluded that, above a low level, expertise and accuracy were unrelated. The book also notes that nonpsychologists have sometimes proved better than psychologists at predicting people’s behavior, complicating the assumption that professional knowledge necessarily confers superior judgment.

Armstrong’s conclusion concerns prediction, not every skilled activity. The book distinguishes forecasting uncertain futures and choosing under uncertainty from work such as auto repair, piloting, skiing, or management, where practice and talent can develop applicable skills. The distinction matters: his critique is directed at claims to general predictive authority, not at specialized competence as such.

Limits of Expert Judgment

The evidence Armstrong discusses points to weaknesses both between and within experts. In fields including stock picking, livestock judging, and clinical psychology, experts agreed with one another less than half the time. A separate study found that medical pathologists’ judgments had an internal consistency of just 0.5: given the same evidence, a pathologist would reach a different opinion half the time. Expertise, in this account, does not guarantee stable judgment even when the information presented is unchanged.

Experts also tend to overestimate how accurate they are. A survey by economist Terrance Odean found this pattern among physicians, nurses, lawyers, engineers, entrepreneurs, and investment bankers. Foreign-exchange traders overestimated the accuracy of their predictions 70 percent of the time; the problem was not only that their forecasts were wrong, but that they lacked a sound sense of how wrong they might be. The book contrasts them with expert bridge players and weather forecasters, whose judgments are routinely well calibrated: when forecasters assign a 30 percent chance of rain, it rains on roughly 30 percent of those days.

The Seer-Sucker Theory

Armstrong summarized his position by arguing that expertise beyond a minimal level is of little value in forecasting change. He also rejected the comforting idea that, even if most experts perform poorly, a few celebrated individuals must be consistently excellent. Claims of accuracy by a single expert, he wrote, would seem to have no practical value. This line of reasoning gave rise to his “seer-sucker theory”: “No matter how much evidence exists that seers do not exist, suckers will pay for the existence of seers.” The phrase names a recurring demand for authoritative forecasters despite the weak evidence for their reliability.

The book does not treat this argument as a reason to disregard sophisticated analysis. Armstrong’s practical implication is that an expert’s advice and predictions should be combined with those of others so that the group can make better use of the information the expert has. A diverse, informed group is expected to outperform most of its members consistently, while an individual expert’s performance can vary sharply from one problem to another. Over time, it is therefore unlikely that one person will reliably surpass the group.

Place in the Book’s Argument

Armstrong’s work reinforces the book’s broader account of collective judgment: groups can be more reliable when their members contribute distinct information and judgments are aggregated, rather than allowing one confident specialist to stand in for the whole. His findings appear alongside examples in which markets and simulated problem-solving groups produce useful forecasts or solutions, but the book does not claim that expertise is irrelevant or that every crowd is wise. Collective judgment still depends on having some information to aggregate and on preserving differences rather than merely multiplying the same error.

The argument is consequently practical as well as skeptical. Because genuinely superior forecasters may exist but are difficult to identify, and past success can reflect luck or measurement error, organizations cannot safely assume that a prestigious track record marks a dependable seer. Armstrong’s research supports pooling informed judgments and resisting the search for a single person who will supply the answer.